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William  M.  McEneaney 
Department  of  Mathematics 
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1  Main  Concepts: 

The  overall  goal  has  been  that  of  computationally  feasible  methods  for  nonlinear  Ro¬ 
bust  and  Hoo  control  and  filtering,  with  the  recent  addition  of  risk-sensitive  control 
and  filtering  (due  to  a  new  result  suggesting  reduced  computations).  The  meth¬ 
ods  are  based  on  max-plus,  min-plus  and  log-plus  algebraic  approaches  to  exact 
linearization  of  the  associated  semi-groups.  Other  tangential  efforts  have  included 
making  use  of  existing  dimensional  reductions  in  certain  problems,  and  a  new  effort 
in  applications  to  command  and  control  for  air  operations. 


2  Overview  of  Results: 

1)  We  have  begun  development  of  max-plus  methods  in  the  solution  of  nonlinear 
Hoo  Bellman  equations  corresponding  to  problems  with  fixed-feedback.  More 
specifically,  we  have  developed  error  estimates  and  convergence  results  for  the 
algorithm.  Software  is  under  development. 

2)  We  have  also  addressed  the  nonlinear  Hoo  control  problem  with  active  control 
computation  in  the  case  where  the  controller  can  dominate  the  disturbance. 
This  leads  to  a  semi-group  which  is  exactly  linear  over  the  min-plus  algebra. 
Software  is  under  development. 

3)  The  linearity  of  certain  nonlinear  risk-sensitive  stochastic  control  and  filter¬ 
ing  problems  under  the  appropriate  log-plus  algebra  has  been  studied,  and 
some  initial  steps  indicating  the  nature  of  algorithms  designed  to  exploit  this 
linearity  have  been  taken. 

4)  The  PI  (with  collaborators  K.  Ito,  Q.  Zhang,  and  W.H.  Fleming)  have  begun 
to  apply  methods  of  robust  control  and  filtering  to  the  problem  for  air 
operations. 

5)  Joint  work  with  J.W.  Helton  and  M.R.  James  on  dimensional  reduction  for 
nonlinear  Hqo  control  under  partial  information  in  the  case  where  there  are 
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some  very-low-noise  measurements  has  been  started.  The  reduction  of  di¬ 
mension  (of  the  space  on  which  the  information  state  must  be  propagated) 
under  exact  measurements  has  been  demonstrated,  and  a  simple  means  of 
compensating  for  the  small  noise  in  those  measurements  is  being  considered. 

6)  Another,  very  interesting,  development  is  that  nonlinear  risk-sensitive  control 
and  nonlinear  Hqo  control  all  take  exactly  the  form  of  standard  stochastic  con¬ 
trol  (i.e.  minimization  of  the  expectation  of  an  integral  cost)  under  changes 
of  algebra  and  probability  measure.  This  work  is  in  a  nascent  state. 

7)  Joint  work  with  M.R.  James  on  application  of  max-plus  methods  to  nonlin¬ 
ear  Hoo  control  under  partial  information  is  underway.  Software  has  been 
developed  by  E.  Gallestey  (a  post-doc)  at  Australian  National  University. 

8)  Joint  work  with  W.H.  Fleming  proving  convergence  of  the  risk-sensitive  filter 
to  the  Hoo  filter  (McEneaney  definition)  has  reached  a  mature  stage.  This  not 
only  proves  a  filtering  result  analogous  to  the  now  well-known  control  result, 
but  also  lends  additional  support  to  the  use  of  the  computationally  preferable 
Hoo  filter  definition  of  McEneaney  (in  contrast  to  definitions  used  by  some 
other  researchers  which  are  actually  more  directly  analogous  to  smoothers  in 
stochastic  approaches). 


3  Detailed  Discussions 


3.1  Numerics  —  Overview  of  Max— Plus  Methods 

We  focused  on  the  nonlinear  Hoo  control  problem  during  the  grant  period.  Consider 
the  case  where  one  has  chosen  a  particular  feedback  form.  Then  the  associated 
semi-group  is  linear  over  the  max-plus  algebra.  This  led  to  a  representation  of 
the  value  function  (available  storage)  as  a  max-plus  eigenvector  of  the  semi-group 
corresponding  to  eigenvalue  zero  (the  max-plus  multiplicative  identity),  that  is 
0  ®  VF  =  Sr[W]  where  W  is  the  value,  Sr  is  the  semi-group,  and  (g)  represents 
max-plus  multiplication.  By  approximating  W  with  a  finite  number  of  functions 
from  a  max-plus  basis  expansion,  the  problem  is  reduced  to  computation  of  a  finite¬ 
dimensional  max-plus  eigenvector,  e,  satisfying  0  ®  e  =  B  ®  e  for  a  particular  B. 

More  recently,  we  began  developing  an  error  analysis  which  demonstrates  that  the 
errors  converge  to  zero  as  the  number  of  vectors  used  in  the  basis  expansion  increases. 
It  is  known  that  there  is  a  unique  max-plus  eigenvalue  for  the  class  of  matrices  to 
which  B  belongs.  We  have  now  demonstrated  that  (in  this  class  of  matrices),  there 
is  also  a  unique  eigenvector  corresponding  to  this  unique  eigenvalue.  We  use  the 
power  method  to  compute  this  eigenvector,  and  guarantee  exact  convergence  in  a 


finite  number  of  steps.  This  quality  of  a  unique  eigenvector  removes  some  previous 
questions  as  to  whether  the  method  would  converge  to  the  correct  solution. 

More  specifically,  we  consider  the  Hoo  problem  for  a  nonlinear  system.  Nonlin¬ 
ear  Hoo  control  and  its  relation  to  the  associated  dynamic  programming  equation 
(DPE)  has  been  studied  extensively  (see  [1],  [11],  [42],  [3]  among  many  notable 
others).  For  continuous  systems,  this  DPE  takes  the  form  of  a  partial  differential 
equation  (PDE).  Unfortunately,  the  Hoo  DPE  is  nonlinear  -  possessing  a  term  which 
is  quadratic  in  the  gradient.  A  significant  recent  development  has  been  the  discovery 
of  new  numerical  methods  exploiting  the  max-plus  (to  be  defined  below)  linearity 
of  the  associated  solution  operator  [18],  [19],  [17],  [8].  These  algorithms  employ  a 
max-plus  basis  expansion  for  the  space  of  semi-convex  functions.  The  max-plus  lin¬ 
earity  was  also  noticed  earlier  in  [12].  In  this  approach,  the  solution  of  the  Hoo  PDE 
is  given  by  a  max-plus  eigenvector  corresponding  to  a  max-plus  eigenvalue  of  0  (the 
max-plus  multiplicative  identity)  for  a  particular  matrix.  This  matrix  is  associated 
with  the  solution  operator  of  the  PDE.  Note  that  the  nonlinear  Hoo  problem  is  trans¬ 
formed  into  a  max-plus  eigenvector  computation  although  one  must  still  compute 
(or  approximate)  the  matrix  defining  this  problem.  This  algorithm  is  analogous  to 
a  spectral  method  for  a  linear  problem  (but  in  the  max-plus  sense)  as  opposed  to 
finite  difference  methods.  We  remind  the  reader  that  the  max-plus  algebra  is  a 

commutative  semi-field  on  3?  U  {— oo}  given  by 

a  ©  6  =  max  {a,  6}, 

a®h  a  +  h 

Note  that  — oo  is  the  additive  identity,  and  0  is  the  multiplicative  identity.  This 
can  be  extended  to  a  field  [2],  but  we  do  not  need  that  here. 

3.2  Details  -  Solution  operator  and  max-plus  linearity 

Consider  the  system 

X  =  f{X)+o{X)w,  X(0)  =  a;  (2) 

where  X  is  the  state  taking  values  in  5R",  /  represents  the  nominal  dynamics,  the 
disturbance  w  lies  in  W  =  {u;  :  [0,  oo)  — >•  JR'"  :  w  €  L2[0,T]  VT  <  oo},  and  a  is  an 
n  X  m  matrix-valued  multiplier  on  the  disturbance. 

We  will  make  the  following  assumptions.  These  assumptions  are  probably  not 
necessary  but  are  sufficient  for  the  results  to  follow.  We  will  assume  that  all  the 
functions  /,  cr  and  I  (given  below)  are  smooth.  We  will  assume  that  there  exist 
K,c  €  (0,  oo)  such  that 

\f{x)-m\<K\x-y\  yx,yeU^ 

(x-vVOix)  -  f  {y))  <-c\x-y\^  Vi.yeS”  (^1) 

m  =  0 


Note  that  the  second  inequality  automatically  implies  the  closed-loop  stability  cri¬ 
terion  of  Hoo  control.  (The  second  inequality  would  not  be  needed  in  the  active 
control  case  to  follow.)  We  assume  that 


\a{x)  -  a{y)\  <  K\x  -  y\ 
\a{x)\  <M  Vx  €  3?" 


for  some  M  <  oo.  Let  l{x)  be  the  running  cost.  We  assume  that  there  exist  /?,  o;  <  oo 
such  that 

\lxx{x)\<^  /.O') 

0  <  l{x)  <  a|xp  Vx  €3?".  ^  ’ 

(There  is  a  reason  for  allowing  ^  to  be  greater  than  2o!,  which  one  might  notice 
below.) 

The  system  is  said  to  satisfy  an  Hco  attenuation  bound  (of  7)  if  there  exists 
7  <  00  and  a  locally  bounded  available  storage  function  W(x),  such  that 

W(x)  =  sup  sup  [  l{X{t)) (3) 

w£WT<ooJo  I 

The  corresponding  DPE  is 

0  =  a{x)a^ {x)VW  -H  f{x)VW  +  /(x).  (4) 

■2x 

It  will  be  assumed  throughout  that  we  are  looking  for  a  solution  satisfying  W(0)  =  0. 
We  will  also  suppose  that  the  above  constants  satisfy 


7^  ^  a 

2^  ^ 


(5) 


We  note  that  there  are  linear  examples  where  (5)  exactly  determines  the  minimum 
disturbance  attenuation  parameter.  One  has  the  following  result  ([26], [30]).  The 
proof  can  be  found  in  the  references. 

Rather  than  dwelling  on  the  PDE  representation  (4) ,  we  would  like  to  show  that 
W  is  a  fixed  point  of  the  solution  operator  to  the  DPE,  where  this  solution  operator 
has  the  representation 

2 

Sr[W{-)]{x)  =  sup{  rZ(X(t))^|u;(t)pdt 

+W(X(t))}. 

By  a  modification  of  the  proof  of  the  previous  theorem,  one  can  show  the  following. 
The  proof  can  be  found  in  [19]. 


Theorem  3.1  For  any  r  €  [0,  oo),  W  given  by  (3)  satisfies  —  W,  and 

further,  it  is  the  unique  solution  in  the  class 

0  <  w(x)  <  (6) 

for  some  6  >  0,  and  this  is  the  “correct”  solution  given  by  (3). 

Note  that  W  is  a.  fixed  point  of  Sj  for  any  r,  which  provides  some  freedom  in 
the  choice  of  problem  we  wish  to  solve.  This  will  be  discussed  further  below.  The 
proof  of  the  max-plus  linearity  of  Sr  is  trivial  (see  [19]). 

Theorem  3.2  The  solution  operator,  Sr,  is  linear  in  the  max-plus  algebra.  That 
is,  (for  constant  c  and  functions  (?!>,  V’j 

<Sr[0  ©  tp]{x)  -  Sr[(l>]{x)  ©  Sr[lp]{x) 

<Sr[c  ®  =  C0Sr[4>]{x) 


3.3  Details  -  Max-plus  basis  and  the  eigenvector  equation 

Define  the  space  of  semi-convex  functions,  S,  as  the  set  of  ^  ^  5R  such 

that  for  any  i?  <  oo  there  exists  Cr  <  oo  such  that  i;{x)  +  ^|a;p  is  convex  over 
Br  =  {x  e  :  |2;|  <  i?}  (see  [18]).  We  refer  to  such  a  ck  as  a  semi-convexity 

constant  for  ^p.  The  proof  of  the  following  is  rather  technical. 

Theorem  3.3  W  lies  in  S. 

The  following  max-plus  basis  can  be  derived  from  convex  duality;  see  [18]  for 
details.  Let  0  e  <S.  Fix  R  <  oo.  Then  (f  is  Lipschitz  continuous  with  some  constant 
Lr  over  Br{0)  (cf.[6]).  Let  {xj}  be  a  countable,  dense  set  over  and  let 

C  >  crI  (see  [18]  for  definition)  be  a  positive  definite  matrix.  Define 

-  Xi) 


for  each  i.  Then, 

OO 

®  [®i  ®  V’i(^)]  y  x  e  Br 

i=l 

where  Oj  =  -maXx^BRi'^iix)  -  <f>{x)].  This  is  a  max-plus  basis  expansion. 
We  assume  C  can  be  chosen  so  that  Sriipfix))  has  an  expansion 


(7) 


(A4) 


i=l 


where 

Bj^i  =  -  m&x{ipj{x)  -  Sr{ipi{x))) 

xeBu 

for  each  i  (we  discuss  this  a  little  more  in  the  next  section).  (This  is  equivalent  to 
an  assumption  that  Sri'ipiix))  is  semi-convex  with  some  constant  on  Br  where 
C  >  c'jil.  In  the  case  where  is  uniformly  nondegenerate,  the  existence  of  can 
be  proven,  but  we  do  not  include  that  here.)  Note  that  B  actually  depends  on  t,  but 
for  this  section  we  fix  any  value  r,  and  suppress  the  dependence  in  the  notation.  In 
order  to  reduce  complexity,  we  suppose  throughout  the  next  two  sections  that  W  has 
a  max-plus  basis  expansion  with  a  finite  number  of  terms.  Let  W  (x)  =  ©"=1  o,i  0 
and  oF  =  (ai  02  •  •  •  On)-  Also  assume  that  the  expansions  of  the  Sr{tpi{x))  terminate 
at  n  terms.  The  full  error  analysis  (where  we  show  that  the  errors  introduced  by 
truncation  go  to  0  as  o  — y  00)  will  be  delayed  to  a  later  paper.  Lastly,  let  us  assume 
that  it  is  required  that 

Cj  >  -00  yj<n,  (A5) 

that  is,  each  basis  function  must  be  active.  The  key  result  is  Theorem  3.4.  The 
proof  appears  in  [17]. 

Theorem  3.4  Sr[W]  =  W  if  and  only  if  a  =  B®a  where  B0a  represents  max-plus 
matrix  multiplication. 

The  proof  appears  in  [13]. 

3.4  Details  -  Convergence  of  the  eigenvector  computations 

Recall  that  we  need  to  solve  the  eigenvector  problem 

0  0  e  =  B  0  e,  or,  e  =  B  0  e.  (8) 

This  requires  two  steps:  computing  (approximately)  B,  and  then  solving  (8)  given 
this  B.  In  this  section,  we  address  the  second  step;  in  the  next  section,  we  will 
address  the  first  step. 

We  will  use  the  power  method  to  compute  the  eigenvector.  That  is,  we  will 
demonstrate  that 

e  =  lim  B'”  0  0 

m—^oo 

where  0  represents  the  zero  vector,  and  the  B^  represents  the  max— plus  product 
repeated  m  times. 


For  the  remainder  of  the  section,  fix  any  r  €  (0,  oo).  Define 

2 

H{x,y)  =  Sr[W]{x)  -  sup  [l{X{t))  -  '^\w{t)\^dt 

+TF(X(r))}  (9) 

where  X(0)  =  0  and  Wj,  =  {t/;  €  W  :  X{t)  =  y}.  We  note  the  following,  but  do  not 
include  the  straightforward  proof  (although  the  machinery  is  in  [30]) 

is  continuous,  (10) 

/f(0,0)  =  0  and  H{x,x)  >  0  if  x  0.  (11) 

Lemma  3.5  Let  w  G  VV. 

ri{X{t))  -  ^|u;(t)l'dt  <  W{x)W{X{r)) 

Jo 

-H(x,X(t)) 

where  W(0)  =  x. 

The  proof  appears  in  [13]. 

Now  let  the  {xj}  be  such  that  Xj  =  0. 

Lemma  3.6  Bi,i  =  0.  Also,  there  exists  5>0  such  that  for  all  j  /  1,  Bjj  <  -5. 


The  proof  appears  in  [13]. 

Theorem  3.7  Let  N  eX,  such  that  1  <  ki  <  n  for  all  i  and  k^+i  =  k^. 

Suppose  we  are  not  in  the  case  N  =  1,  ki  =  k2  =  f-  Then 

N 

Bki,ki+i  ^ 
i=l 


The  proof  appears  in  [13]. 

We  should  note  that  B  has  a  unique  eigenvalue,  although  possibly  many  eigen¬ 
vectors  corresponding  to  that  eigenvalue  [2].  By  the  above  results,  this  eigenvalue 
must  be  zero  (ignoring  errors  due  to  approximation). 

Theorem  3.8  lim/vn-oo  B^  <2>0  exists,  converges  in  a  finite  number  of  steps,  and 
satisfies  e  =  B  ®  e. 


The  proof  appears  in  [13]. 

Not  only  is  the  eigenvalue  unique,  but  we  can  also  show  that 

Corollary  3.9  There  is  a  unique  eigenvector  up  to  a  max-plus  multiplicative  con¬ 
stant,  and  of  course,  this  is  the  output  of  the  above  power  method. 


The  proof  appears  in  [13]. 


3.5  Details  -  Approximation  of  B 


The  feasibility  of  the  algorithm  is  dependent  upon  a  feasibile  approximation  algo¬ 
rithm  for  B.  One  approach  is  a  Taylor  series  (in  t)  approximation  to  St[4>i]{x).  More 
specifically,  letting  V{t,x)  =  St[ipi]{x),  so  that  V  satisfies 

=  f  .VV  +  1  +  VV^aa'^VV  (12) 

y(0,a;)  =  ipi{x) 

one  may  approximate  V  as 

V{t,  x)  =  Vo(3:)  +  Viix)t  +  ^V2{x)t^  +  ....  (13) 

Here  Vo{x)  =  ipiix)  and  Vi  is  the  right  hand  side  of  (12)  with  replacing  The 

higher  order  terms  are  computed  by  differentiating  (12),  and  we  do  not  include  them. 
Then  ^ 

Bj^i  =  -  m^  ^ipjix)  -  [Vo  +  ViT  -1-  +  •  •  -Kx) 

This  method  was  applied  but  suffers  from  a  problem  which  we  describe  only 

briefly.  Note  that  the  approximation  of  V  via,  say  three  terms,  in  the  Taylor  series 
at  X  out  to  time  r  can  be  improved  by  reducing  r.  However,  the  argmax  moves  off 
toward  “oo”  as  r  4-  0,  and  the  Taylor  series  approximation  degrades  as  x  moves  off 
to  oo!  Consequently,  the  approximation  of  Bj^i  does  not  tend  to  improve  as  r  J.  0, 
and  so  this  approach  was  abandoned. 

In  its  stead  we  are  using  a  means  of  approximately  tracking  the  argmax  for  small 
time  intervals.  Let 

where 

^(t)  =Xi  +  {t/T){Xj-Xi). 

Let  V{t,x)  be  as  above  (with  ipi  as  initial  condition).  Let 

X(t)  =  argmax|V'j(i,a:)  -  V'(t,a;)| . 


Note  that 


X{t)  =  argmax  {^j(x)  -  'H(r,a:)} 


which  is  the  desired  quantity.  The  replacement  of  ipj{-)  by  ipjit,-)  prevents  the 
argmax  from  “blowing  up”  at  <  0.  We  use  the  Taylor  expansions 


v{t,x)mt)+vit){x-x{t)) 

+i(i  -  X{t)fV2(t){x  -  X{t))  +  ... 


(where  the  k  subscripts  in  the  /  expansion  indicate  components).  One  then  obtain 
the  following  sequence  of  ordinary  differential  equations  for  the  propagation  of  X 
leading  to  a  computation  of  Bj^ii 

Xk  =  ~  C)  —  ~  ^j[) 

i 

+  fl0^2xk,x,) 

I 

-‘^Yl(^kx,0-l,mV2xi,Xk)  +  2Xfc]| 
l^m 

Vo  =  +  Xk- 

k  I 

plus  higher  order  equations  which  we  do  not  include  here.  It  will  be  shown  that  the 
error  induced  by  cutting  off  this  series  at  a  finite  number  of  differential  equations 
can  be  bounded  for  small  time.  We  also  note  that  the  initial  conditions  for  this 
propagation  are  easily  obtained,  although  we  do  not  include  them  here. 

Two  examples  appear  in  figures  1  and  2.  In  the  first,  a  linear-quadratic  example 
is  considered.  Both  the  exact  solution,  and  the  numerically  computed  solution  are 
displayed.  In  figure  2,  the  solution  of  a  nonlinear  problem  is  displayed.  Although 
there  is  a  significant  nonlinearity,  the  solution  is  still  rather  well-behaved;  less  well- 
behaved  examples  will  need  to  be  generated.  Each  computation  took  less  than  10 
seconds  on  a  Unix  Ultra  10  workstation. 

There  still  remain  some  significant  questions,  such  as  how  to  reasonably  choose 
the  basis  functions  and  time-step. 

Further  information  can  be  found  in  [13],  [17],  [19]  and  [18]. 


3.6  Numerics  -  Min-Plus  Methods 


Min-plus  methods  are  being  used  for  the  solution  of  nonlinear  Hqo  control  problems 
where  there  is  active  control  computation  (as  opposed  to  testing  of  feedbacks  above) . 
For  instance,  consider  the  problem  with  dynamics  X  =  A{X)  +  B{X)u  +  a{X)'w 
and  cost  2  2 

J  =  I"  L{X{t))  +  |-|u(t)|'  -  Y\w{t)\Ut 

where  u  is  the  control,  w  is  the  disturbance  and  X  is  the  state.  The  corresponding 
Isaacs  equation  takes  the  form 


0  =  A{x)  •  VW  +  L{x)  - 


'B{x)B'^{x) 

2772 


cr{x)a'^{x) 

V 


vw. 


If  u  dominates  w  in  the  sense  that  the  Hamiltonian  is  concave,  then  the  associated 
semi— group  is  linear  over  the  min— plus  algebra.  All  the  results  of  item  1  have 
analogues  in  this  case.  Software  for  two-dimensional  problems  is  being  debugged. 


Further  information  can  be  found  in  [13]  and  [17]. 


3.7  Min-Plus  Example 

As  an  example,  we  consider  the  min-plus  problem  in  two-dimensions  with  X  {t)  - 
{xi{t),X2{t)Y  and  dynamics 

^2 )  ""  ( (3/4)  arcLn(2xi) )  "^  ( 1 )  “  '^  ( 1 ) 

Let  7, 77  be  such  that  the  reduction  of  the  previous  section  yields 

QQ^  _  ro  0  ] 

2  “  [0  1/2.  ■ 

Note  that  this  example  was  chose  so  as  to  represent  a  second-order  system  of  the 
form 

y  =  (3/4)  arctan(2j/)  +  u  +  w. 

The  running  cost  was  simply  1{X)  =  x\  +  x\. 

The  computations  were  run  to  obtain  the  results  depicted  in  the  figures  below. 
The  computations  were  run  at  a  coarseness  level  that  allowed  computation  in  less 
than  10  seconds  on  a  Sun  Ultra  10.  Figure  1  depicts  W.  Figures  2  and  3  depict  the 
partials  of  W;  note  the  non-smoothness.  Figure  4  depicts  the  approximate  error 
computed  by  taking  first-order  differences  (probably  too  coarse  a  method)  on  the 
grid  to  approximate  VVF,  and  then  plugging  this  back  into  the  PDE.  The  resulting 
values  were  scaled  by  dividing  by  1  +  |a:p. 


Figure  1:  Storage  function 


Figure  2:  Partial 


Figure  3:  Partial 

3.8  Numerics  —  Log— Plus  Methods 

Nonlinear  risk-sensitive  problems  such  as  risk-sensitive  filtering  exhibit  the  same 
type  of  behavior  as  described  in  the  first  item  in  the  case  of  Hqo  problems.  Specif¬ 
ically,  there  exist  a  set  of  algebras  (actually  fields  in  this  case)  indexed  by  a  pa¬ 
rameter  e,  such  that  the  risk-sensitive  problems  have  semi-groups  which  are  linear. 
The  question  of  how  to  choose  an  appropriate  basis  to  exploit  this  linearity  remains 
open. 

Further  documentation  can  be  found  in  [16]. 


-«  -1 

Figure  4:  Scaled  back-substitution  error 

3.9  Theory/Numerics  -  Dimensional  Reduction 

Since  filtering  and  control  under  partial  observations  can  quickly  become  compu¬ 
tationally  unwieldy  as  the  dimension  increases,  we  consider  the  case  where  the 
measurements  are  such  that  one  might  know  some  components  of  the  state  al¬ 
most  exactly.  This  has  been  termed  the  “cheap  sensor”  case.  We  showed  that  if 
m  state-components  were  measured  exactly  in  a  problem  with  n  state-components, 
the  computation  of  the  information  state  could  be  reduced  to  propagation  on  an 
affine  n-m  dimensional  space,  with  this  affine  space  moving  according  to  the  exact 
measurements.  This  was  been  documented  in  a  CDC  paper,  and  a  journal  paper 
has  been  submitted  to  IEEE  TAG.  However,  one  needs  to  ensure  that  the  method 
is  robust  to  small  errors  in  the  nearly  exact  measurements.  We  are  now  beginning 
a  study  of  this  problem.  We  will  show  that  a  “steep”  quadratic  in  the  m  state 
components  will  be  the  next  step  in  an  expansion  around  the  exact  measurement 
situation.  (Joint  work  with  J.W.  Helton.) 

Further  documentation  can  be  found  in  [10]  and  [20]. 

3.10  Theory  -  Risk-Sensitive  Filtering 

It  has  been  demonstrated  that  a  risk-sensitive  filter  converges  to  a  Robust/Hoo  filter 
(McEneaney  definition).  This  result  is  directly  analogous  to  similar  results  which 
we  have  obtained  in  recent  years  for  the  control  problem.  This  also  adds  further 
weight  to  the  appropriateness  of  the  (McEneaney  definition  of  the)  Hoo  filter  for 
tracking  applications  (as  opposed  to  more  computationally  difficult  definitions  be¬ 
ing  employed  for  partially  observed  control  which  are  actually  more  properly  termed 


smoothers)  (Joint  work  with  W.H.  Fleming.) 

Further  documentation  can  be  found  in  [7]  and  [24]. 


3.11  Theory /Numerics  -  Partially  Observed  Hqo  Control 

In  joint  work  with  M.R.  James,  we  have  developed  an  approach  to  Hoo  control  under 
partial  information  where  the  information  state  is  propagated  via  max-plus  meth¬ 
ods.  Software  has  been  developed  at  Australia  National  University  with  the  aid  of 
a  post-doc. 

Further  documentation  can  be  found  in  [15]. 


3.12  Theory  -  General 


We  showed  that  nonlinear  risk-sensitive  problems  and  Hoo  problems  take  exactly  the 
same  form  as  traditional  stochastic  problems  -  once  one  transforms  the  underlying 
algebra.  That  is,  in  traditional  stochastic  control  problems,  one  attempts  to  min¬ 
imize  an  expectation  of  some  integral  cost  criterion  such  as  E{Jq  L{X{t))dt}.  A 
corresponding  risk-sensitive  criterion  would  take  the  form 


elog 


E 


L{X{t))dt 


(2) 


while  an  Hoo  criterion  would  take  the  form 

sup  [  [L{X{t))  -  \w{t)\‘^]dt. 
weL^ Jo 


(3) 


Now,  employ  the  appropriate  log-plus  algebra  for  criterion  (2),  and  the  max-plus 
algebra  for  criterion  (3).  Next,  adjust  the  definition  of  a  probability  measure  to 
coincide  with  the  algebra.  For  instance,  the  probability  measure,  P',  must  satisfy 
la  <  P'{A)  <  Im  for  any  measurable  A  where  la  and  Im  are  the  additive  and 
multiplicative  identities.  (In  the  standard  algebra,  this  is  simply  0  <  P{A)  <  1.) 
Then  criteria  (2),  (3)  take  the  (traditional  stochastic)  form 

E^  L{Xit))dty 

where  the  e  superscript  indicates  a  probability  measure  corresponding  to  the  appro¬ 
priate  log-plus  or  max-plus  algebra.  As  e  i  0,  the  log-plus  algebras  converge  to 
the  max-plus  algebra,  and  the  log-plus  probability  measures  converge  to  the  max- 
plus  measure.  Consequently,  one  can  also  interpret  the  convergence  of  risk-sensitive 
control  to  Hqo  control  in  terms  of  convergence  of  the  algebras  and  corresponding 
probability  measures.  This  is  a  major  re-interpretation.  This  work  is  in  its  infancy. 

Further  documentation  can  be  found  in  [16]. 
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